Since 1991, when the PCI bus was introduced as one of the first industry standards for input/output and peripheral communications, most computer system architectures have utilized PCI for input/output (I/O) communications. During the time that the PCI standard has gained wide acceptance, the performance capabilities of the typical computer system have drastically improved. The increases in computer performance and the associated demands placed upon computer networks by end users have rapidly exceeded the capabilities for which the PCI bus was designed. Expensive high-end servers, database access systems, and network switches have addressed the shortcomings of the PCI standard through the creation of specialized and proprietary data transfer architectures. Newer, faster versions of the PCI architecture are also common, but these have the same inherent disadvantage of the PCI bus in that a parallel architecture is used. With the anticipated continued growth of demand for data services over the internet and in computer-computer communications, a new long-term solution was needed.
One proposed solution is the Infiniband architecture. Infiniband is a joint effort of several computer and peripheral manufacturers, including IBM, Intel, Compaq, HP, Dell, Microsoft and Sun Microsystems. The Infiniband specification describes an interconnect technology that enables processor nodes and input/output nodes to be joined together to form a system area network. The Infiniband architecture, if properly designed, is independent of the type of operating system and the processing platform used. This architecture is partially based upon the observation that with the needs for greater bandwidth and lower data latency, the input/output devices in a network have become more complex and have increased processing capabilities.
Referring to FIG. 1, a high-level diagram of one subnetwork of the Infiniband Architecture 100 is shown. The Infiniband architecture is designed using an extensive I/O switched network. The subnetwork shown in FIG. 1 uses a point-to-point switching network, where each switch may be linked to one or more processing nodes, routers, I/O devices, and data storage devices. This type of link connectivity may be characterized as module-to-module or chassis-to-chassis, depending upon whether the links are within a larger subsystem or are links between subsystems. The collection of switches is called the Infiniband switching fabric, or just fabric. The fabric is the heart of the Infiniband architecture and processing devices may be coupled to this fabric in a variety of ways. Typically each device coupled to the fabric has a channel adaptor interface. The Infiniband specification specifies two types of channel adaptors; the Host Channel Adaptor (HCA) and the Target Channel Adaptor (TCA). The HCA is designed for nodes that require more processing capability. The TCA is typically designed to support input/output devices. Both the TCA and HCA manage the transport level interactions with the switching fabric.
Referring now to FIG. 2, a more detailed diagram of a channel adaptor is shown, according to the prior art. Channel adaptors are used in the Infiniband architecture to generate and consume packets. These channel adaptors are present in the form of TCA's and HCA's. A channel adaptor allows devices external to the Infiniband specification access to data through a programmable direct memory access (DMA) engine. This DMA access may be executed locally or remotely. A channel adaptor may have multiple ports as shown in FIG. 2. Each port is coupled to the transport functionality using one or more Virtual Lanes (VLs). Each VL provides it's own buffering flow control so that each port can send and receive data packets simultaneously.
Referring now to FIG. 3, a more detailed diagram of an Infiniband switch is shown, according to the prior art. The switches specified in the Infiniband architecture generate or consume data. Packets received by the switch are forwarded to the destination address specified in the packets routing header. Thus, the utility of switches in the Infiniband architecture is the ability to interconnect links using switches to relay packets between the links. Switches are not directly addressed from the point of view of the end user. Instead, packets traverse a subnetwork transparently through the use of unique identifiers (LIDs). The switch forwards individual packets to an outgoing port using the value specified in the LID. One or more ports that may be used for incoming and outgoing relay functionality are coupled to the switch through one or more Virtual Lanes. In general, Infiniband switches support unicast or multicast routing. In unicast routing, a single packet is delivered to one destination, while multicast routing delivers a single packet to multiple destinations.
In many switching implementations, a packet is transferred out of a switch using a multiple-cycle request/arbitration/grant operation. During the request phase, the switch queries a destination port, with a request to transfer the packet. The arbitration phase begins with the switch and the destination port exchanging transport information. Based upon the transport parameters of the packet, the packet is granted with a certain quality of service level. From the point of view of the destination port, the arbitration process is a method to handle multiple incoming requests received and to select the appropriate requests that will be granted. If the two sides can agree on the parameters for transport of the packet, the grant phase begins and the switch starts the process of sending the packet to the destination port.
If the packet has a requirement for low-latency, then the entire portion of the request/arbitration/grant cycle that does not involve the actual transmission of the packet to the destination port increases the latency of the packet. Techniques for minimizing this latency include complex arbitration algorithms as well as techniques that start the transport of the packet while the packet is still being received by the switch.
In latency-sensitive systems, it is highly desirable to have switches operate in cut-thru mode. That is, switching logic begins transferring a packet out of a switch while the switch is still receiving the packet. This becomes a difficult problem when protocol requires that error packets be discarded or marked bad, while performance requires low latency packet transfers. There is a need in the art for an approach that is able to handle and discard packets even if the switch may have not received the entire packet and checked it for errors prior to starting the request/arbitration/grant cycle that sends the packet to the destination port.